Smart session load balancer and throttle

ABSTRACT

A method of processing packets propagated over a packet switched communications network having a control plane, user plane, and a plurality of probes, the method comprising: receiving at least one control plane packet associated with creating at least one user session in the network; selecting a set of user sessions from the at least one user session; determining at least one target feature that characterizes packets propagated over the network; and load balancing all packets sharing the at least one target feature that belong to a same user session of the set of user sessions to a same probe of the plurality of probes for processing by the probe.

RELATED APPLICATIONS

The present application is a continuation in part of U.S. patentapplication Ser. No. 16/102,842 filed on Aug. 14, 2018, which is acontinuation in part of U.S. patent application Ser. No. 15/325,462filed on Jan. 11, 2017, now U.S. Pat. No. 10,050,886 issued on Aug. 14,2018, which is a U.S. National Phase of PCT ApplicationPCT/IB2016/054332, filed on Jul. 21, 2016, which claims the benefitunder 35 U.S.C. 119 (e) of U.S. Provisional Application 62/334,170 filedon May 10, 2016, the disclosures of which are incorporated herein byreference.

FIELD

Embodiments of the disclosure relate to load balancing tasks comprisedin monitoring a communication network.

BACKGROUND

Modern communication networks support large, varied, and growingpopulations of users and an ever-increasing gamut of user services anduser applications for users who access the networks using any of variousstationary and/or mobile user equipment (UE). The networks span not onlyglobal populations and geography, but a plethora of methods, devices,infrastructures, and protocols for propagating data, voice, and videocontent provided by a host of different large and small serviceproviders.

As the various software and hardware resources that support thenetworks, configure paths along which network packets propagate in thenetworks, and provide services mediated by the networks have become moresophisticated, the resources, which may be referred to as networkequipment (NE), have become fungible and logically abstracted away fromtheir particular physical structures. The physical devices and apparatusunderlying modern communication networks have become commodity resourcesthat are configurable by software to provide virtualizations of networkfunctions, referred to as “Network Functions Virtualization” (NFV), and“Software Defined Networks” (SDNs). And dedicated physical devices thatprovide particular network functionalities have been or are beingreplaced by software entities that access the physical commodities of anetwork, and/or other software entities of the network, on an “as need”basis to communicate with each other and provide functionalitiesrequired by the networks. The zoology of software entities areconventionally referred to as virtual entities, with a particularvirtual entity typically distinguished by a name of a functionality itprovides.

The extensive, complex infrastructure of many modern communicationnetworks typically comprise a mix of dedicated physical devices and anever increasing, if not dominant number, of virtual network entitiesengaged in an incessant packet exchange chatter approaching hundreds ofgigabits per second (Gbps). Monitoring and troubleshooting the networksas much as possible in real time to maintain performance that providesacceptable quality of service (QoS) and user quality of experience (QoE)for the multitude of services that the networks provide has become ademanding challenge.

SUMMARY

An aspect of an embodiment of the disclosure relates to providing anoptionally virtualized load balancing system that is relatively easilyscalable and configured to efficiently distribute data propagating in apacket switched communication network to probes and/or other networkmonitoring entities, such as network security devices, networkanalyzers, and other network monitoring tools, generically referred toas probes, that operate to process the data and provide the network withinformation relevant to understanding, monetizing, and/or controlling,optionally, network performance. In an embodiment, the virtualized loadbalancing system, which may be referred to as a “Smart-vLB system” orsimply a “Smart-vLB”, comprises a hierarchical architecture comprisingat least one “non-stateful” virtual load balancer (NS-vLB) and at leastone stateful virtual load balancer (S-vLB). The at least one NS-vLB andat least one S-vLB cooperate to distribute data in packets that theSmart-vLB receives from the network for processing, to a plurality ofprobes, optionally comprising virtual probes (vProbes). In anembodiment, the communication network comprises an evolved packet system(EPS) network, which as discussed below may also be referred to as anLTE (long term evolution) network.

The at least one NS-vLB in the Smart-vLB receives packets, which may becontrol plane and/or user plane packets, that may be mirrored from atleast one port in the network and distinguishes between packets,optionally referred to as “contextual packets”, that it receives, andpackets, optionally referred to as “non-contextual packets”, which itreceives. A contextual packet is a packet comprising data that may notbe processed for a given desired purpose without reference to data fromanother, associated contextual packet and may advantageously beprocessed by a same virtual probe that processes data from the otherassociated contextual packet. For example, a contextual packet maycomprise data that may not be directly associated with a session ortransaction to which it belongs but may be associated with another,associated, packet to determine a session or transaction to which thecontextual packet belongs. Packets may be deemed associated by sharing acommon attribute, such as by way of example, sharing a same networkprotocol or application, and/or belonging to a same network session ortransaction. A non-contextual packet, is a packet comprising data thatmay be processed for a given desired purpose without reference to datafrom another packet and may for the given desired purpose be processedby a probe that is different from a probe that processes other packetsfor the same desired purpose. A non-contextual packet may by way ofexample be a packet that may be directly associated with a session ortransaction to which it belongs without reference to data in anotherpacket. It is noted that a given packet may be contextual for a givenparticular purpose but non-contextual for a different particular givenpurpose.

In an embodiment, each of the at least one NS-vLB is configured totransmit non-contextual packets that it receives for processing to agiven physical or virtual probe of optionally a plurality of physical orvirtual probes without reference to information in another packet. TheNS-vLB is configured to transmit associated contextual packetscharacterized by a same shared attribute to a same S-vLB. The S-vLBtransmits contextual packets for processing to a vProbe having anidentity which the S-vLB determines based on information from at leastone of a plurality of other packets that it receives.

In an embodiment the Smart-vLB is configured to load balance packets andcontrol, “throttle”, transmission rates of copies of packets to a probefor processing to prevent oversubscribing probe capacity. Optionally,load balancing and throttling transmission rates comprise associatingthe packets with a particular feature, also referred to as a targetfeature, that characterizes the packets and load balancing andthrottling based on the particular feature. In an embodiment the targetfeature of the packets may by way of example, be an IMSI or an IMEI(International Mobile Equipment Identity) of a UE associated withtransmission of the packets, an MSISDN (Mobile Station InternationalSubscriber Directory Number), APN (Access Point Name), QCI, (Quality ofService Class Identifier), and/or a CGI (Cell Global Identifier).

In an embodiment, the Smart-vLB operates to load balance and/or throttletransmission of network packets by selecting a limited number of usersessions from which packets propagated over the network are forwarded toprobes, and forwarding substantially all, or a particular selection ofpackets belonging to a same given selected user session to a same probedesignated to process packets from the given selected user session. Auser session is a temporary communication interchange supported by thenetwork from a time at which a control plane Create SessionRequest/Response transaction creates the session to a time at which acontrol plane Delete Session Request/Response transaction deletes thesession. A given user session is associated with a particular IMSI andparticular MS-IP (mobile subscriber IP) address and includes all userplane packets propagated over the network addressed from or to the MS-IPbetween the times at which the control plane Create Session and DeleteSession transactions respectively create and delete the user session.Optionally, control plane messages of the control plane Create SessionRequest/Response transaction and the Delete Session Request/Response areGTP-C messages.

A probe that processes packets from a given user session may be said to“process the session”, and a monitored session is a session for whichsubstantially all or a particular selection of the session packets areload balanced to a same probe for processing. Load balancing packetsresponsive to their respective user sessions in accordance with anembodiment of the disclosure so that all or a particular selection ofpackets belonging to a same session are load balanced to a designatedsame probe, may be referred to as session integrity load balancing(SILBA or SILBA load balancing). Throttling packets responsive to theirrespective user sessions to load balance packets belonging to a limitedselection of user sessions in accordance with an embodiment may bereferred to as “session integrity throttling” (“SIT” or “SITthrottling”).

In an embodiment the Smart-vLB monitors control plane packets propagatedby the network to determine when user sessions are created and deleted.The Smart-vLB operates to identify control plane packets associated witha Create Session Request/Response transaction and/or subsequent ModifyBearer Request/Response transaction associated respectively withcreating and subsequently modifying a same given user session. TheSmart-vLB process the identified control plane packets to determine aset of target features, also referred to as session ID features, thatmay be used to determine whether a packet propagated over the networkbelongs to the given user session. To SILBA load balance user sessionpackets propagated over the network Smart-vLB uses the session IDfeatures for a given user session to identify propagated packets thatbelong to the given user session and load balances the identified userpackets to a same probe. To SIT throttle user session packets Smart-vLBselects a limited number of created user sessions from which to processpackets, and uses the session ID features for a given selected usersession to identify packets propagated over the network that belong tothe given selected user session. Packets identified as belonging to thegiven selected user session are load balanced. A Smart-vLB in accordancewith an embodiment of the disclosure may SIT throttle, may SILBA loadbalance, or both SIT throttle and SILBA balance control plane and/oruser packets. It is noted that SIT throttling and SILBA load balancingare commutative operations.

In an embodiment Smart-vLB determines a fraction less than one,hereinafter also referred to as a selection fraction, of created usersessions from which to forward packets for processing. The Smart-vLBselects created sessions for monitoring or packet processing, so that anumber of selected user sessions is substantially equal to the selectionfraction of created user sessions. Optionally, selection is random. Inan embodiment, a selection fraction may be applied to a group of createduser sessions that are characterized by a range of values for aparticular target feature. A selection fraction may be changedresponsive to magnitude of network traffic.

In an embodiment, if a given probe of network probes that operate toprocess packets from selected user sessions is to be retired, Smart-vLBstops load balancing newly selected user sessions to the given probe andload balances newly selected sessions to other probes of the networkprobes. The given probe is allowed to continue processing user sessionsin a particular set of selected user sessions for which the given probewas responsible prior to retirement, but no new user sessions areassigned to the given probe for processing. As a result, a number ofuser sessions that the given probe processes decreases as user sessionsin the particular set of user sessions are closed. When a last usersession of the particular set of user sessions is closed, the givenprobe is retired. When a new probe is instantiated, from a time at whichthe new probe becomes functional, Smart-vLB includes the new probe whenload balancing newly selected sessions for processing to the probes. Fora situation in which a given probe fails, at least one surrogate probemay be assigned to process user sessions for which the failed probeprocessed packets before failure. The at least one surrogate probe maybe at least one functioning network probe “remaining” after the givenprobe failed, or at least one new probe that was instantiated as part ofan automated or manual healing process in response to failure of thegiven probe. As of a time at which the surrogate probe is assigned andoperative the surrogate probe is included among functioning networkprobes to which, Smart-vLB load balances user sessions. Optionally, usersessions processed by a failed probe prior to failure are evenly loadbalanced between probes that are functioning after the failure.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter.

BRIEF DESCRIPTION OF FIGURES

Non-limiting examples of embodiments of the disclosure are describedbelow with reference to figures attached hereto that are listedfollowing this paragraph. Identical features that appear in more thanone figure are generally labeled with a same label in all the figures inwhich they appear. A label labeling an icon representing a given featureof an embodiment of the disclosure in a figure may be used to referencethe given feature. Dimensions of features shown in the figures arechosen for convenience and clarity of presentation and are notnecessarily shown to scale.

FIG. 1 schematically shows a long-term evolution, LTE, communicationsnetwork;

FIG. 2 schematically shows a Smart-vLB comprising NS-vLBs and S-vLBsoperating to load balance processing of data comprised in controlpackets transmitted by network entities of the LTE shown in FIG. 1 , inaccordance with an embodiment of the disclosure;

FIG. 3 shows a flow diagram of a procedure by which the NS-vLBscomprised in the Smart-vLB shown in FIG. 2 distribute packets that theSmart-vLB receives from the LTE network, in accordance with anembodiment of the disclosure;

FIGS. 4A and 4B show a flow diagram of a procedure by which one of theS-vLBs comprised in the Smart-vLB shown in FIG. 2 distribute packetsthat it receives from a NS-vLB in the Smart-vLB, in accordance with anembodiment of the disclosure; and

FIGS. 5A-5C show a flow diagram of a procedure by which another of theS-vLBs comprised in the Smart-vLB shown in FIG. 2 distribute packetsthat it receives from a NS-vLB in the Smart-vLB, in accordance with anembodiment of the disclosure;

FIG. 6 schematically shows a Smart-vLB comprising NS-vLBs and S-vLBsoperating to load balance processing of data comprised in controlpackets propagated to and from a Serving Gateway (SGW), and/or a PacketData Network Gateway (PGW) of the LTE shown in FIG. 1 , in accordancewith an embodiment of the disclosure;

FIGS. 7A and 7B show flow diagrams of procedures by which and NS-vLB andan S-vLB comprised in the Smart-vLB shown in FIG. 6 operate todistribute packets, in accordance with an embodiment of the disclosure;

FIG. 8 shows a flow diagram of a procedure by which the Smart-vLB shownin FIG. 6 may operate to load balance and throttle transmission rates ofpackets transmitted to probes for processing, in accordance with anembodiment of the disclosure; and

FIG. 9 shows a flow diagram of a procedure by which a Smart-vLB mayperform session integrity throttling (SIT), in accordance with anembodiment of the disclosure.

DETAILED DESCRIPTION

In the following detailed description, a Smart-vLB that is configured inaccordance with an embodiment of the disclosure to load balanceprocessing of control data transmitted in packets by entities comprisedin an LTE communications network is described with reference to FIGS.1-9 . Features and functioning of the LTE network are discussed withreference to FIG. 1 . FIG. 2 schematically shows the Smart-vLB operatingto load balance processing of data in control plane packets transmittedto and/or from an MME comprised in the LTE network shown in FIG. 1 .FIG. 6 schematically shows the Smart-vLB operating to load balanceprocessing of data in control and/or user plane packets transmitted toand/or from an MME comprised in the LTE network shown in FIG. 1 .Whereas the Smart-vLB in FIG. 2 and FIG. 6 is shown by way of examplecomprising two NS-vLBs and two S-vLBs, practice of an embodiment of thedisclosure is not limited to Smart-vLBs comprising two each of NS-vLBsand S-vLBs. A Smart-vLB in accordance with an embodiment of thedisclosure may comprise more or less than two NS-vLBs and more or lessthan two S-vLBs. Procedures by which component NS-vLBs and S-vLBs in theSmart-vLB may process packets that the Smart-vLB receives from the LTEnetwork are discussed with reference to the flow diagrams shown in FIGS.3A-5C, 7A-7B, 8 and 9 .

In the discussion, unless otherwise stated, adjectives such as“substantially” and “about” modifying a condition or relationshipcharacteristic of a feature or features of an embodiment of thedisclosure, are understood to mean that the condition or characteristicis defined to within tolerances that are acceptable for operation of theembodiment for an application for which the embodiment is intended.Wherever a general term in the disclosure is illustrated by reference toan example instance or a list of example instances, the instance orinstances referred to, are by way of non-limiting example instances ofthe general term, and the general term is not intended to be limited tothe specific example instance or instances referred to. Unless otherwiseindicated, the word “or” in the description and claims is considered tobe the inclusive “or” rather than the exclusive or, and indicates atleast one of, or any combination of more than one of items it conjoins.

FIG. 1 schematically shows network elements comprised in a mobilecommunication network 18 comprising a 3rd Generation Wireless MobileCommunication (3G) network 19 and an evolved packet system (EPS)communications network 20 that operates to connect a user's mobilecommunication equipment (UE), to one or more Packet Data Networks(PDNs), and/or, as a circuit switch fallback option, to the PublicSwitched Telephone Network (PSTN). Network 20 may operate to provide auser's UE with communications to the Internet and/or to stationaryand/or mobile communication equipment of other users (not shown). InFIG. 1 mobile user equipment, UE, is schematically represented bysmartphone and laptop icons 22.

EPC network 20 comprises a packet switched core network referred to asan Evolved Packet Core (EPC) core network indicated by a dashed boundarylabeled EPC and a Long Term Evolution (LTE) radio access networkreferred to as an Evolved Universal Terrestrial Radio Access NetworkE-UTRAN indicated by a dashed boundary labeled E-UTRAN. E-UTRAN providesuser equipment with wireless radio frequency (RF) channel connectivityto the core network EPC. The core network EPC provides a UE connected tothe core network via an E-UTRAN radio channel with a communicationsconnection to a PDN via an internet protocol (IP) packet flow“pipeline”, conventionally referred to as a “bearer”. The acronym LTE isconventionally used when referring to an evolved packet system (EPS)network comprising both the LTE E-UTRAN and the evolved packet core EPC,and network 20 shown in FIG. 1 may be referred to as LTE network 20, orsimply LTE.

E-UTRAN comprises a network of RF communication base stations referredto as evolved NodeB (eNodeB) base stations. Each eNodeB comprises an RFtransceiver and integral controller that controls the RF transceiver toestablish radio communication channels between UEs in a limitedgeographic region referred to as a cell, and the core network EPC.Legacy base stations, referred to as NodeB base stations used in 3Gnetworks, do not comprise integral controllers but are connected toradio network controllers (RNCs), operate using RF transmissiontechnologies different from those used by eNodeBs and have limitedfunctionalities in comparison to eNodeB base stations. Core network EPCis capable of communicating with and connecting a UE connected to aNodeB with a PDN network, and LTE network 20 is schematically shownconnected to a NodeB base station of 3G network 19 via an RNC and aServing GPRS (General Packet Radio Service) Support Node, (SGSN) of the3G network. The SGSN node is responsible for delivery of packets fromand to UEs that are connected to a NodeB base station rather than aneNodeB base station. The SGSN is connected to the Internet via a GGSN(GPRS Gateway Support Node). Whereas in FIG. 1 mobile network 18 isshown comprising a E-UTRAN having two eNodeBs and a 3G network havingone NodeB, an LTE may and typically does comprise more than two eNodeBsand a 3G network comprising more than one NodeB.

Core network EPC of LTE 20 comprises a plurality of logical nodes, whichare supported by virtual and/or physical network entities. The networknodes include, a Serving Gateway (SGW), a PDN gateway (PGW), a HomeSubscriber Server (HSS), a Mobile Switching Center (MSC) server, and aMobile Management Entity (MME). The SGW is a node of the LTE networkthrough which data packets from a UE connected to an eNodeB enter theEPC to be propagated from the UE through the EPC to their destinations,or exit the EPC to propagate to the eNodeB and the UE after propagatingthrough the EPC from their sources. The SGW serves as a local mobilityanchor for data bearers when the UE moves between cells of differenteNodeBs. The PGW is a node through which UE data packets from a UEconnected to the EPC exit the EPC to propagate to a PDN or enter the EPCfrom a PDN to propagate to the UE. The PGW is responsible for allocatingan IP address to the UE and handling various administrative and chargingtasks in accordance with information received by the PGW from a nodereferred to as a Policy and Charging Rules Function (PCRF) node. The HSSnode contains user subscription data, such as a QoS profile and anyrestriction that might limit user roaming. The HSS also contains dynamicinformation such as an identity of an MME to which the user UE maycurrently be connected. The MSC node supports signaling between an MMEand a UE Subscriber Identity Module (SIM) and Short Message Services(SMS) for a UE connected to the EPC via a NodeB.

The MME is a central control and signaling node that communicates withand orchestrates the activities of the other nodes and the UE inmanaging the resources of the LTE to establish, route, and maintain IPpacket flow between a UE and a PDN. An EPC generally comprises more thanone MME and may hand over connection to a UE from one to another of theMMEs as the UE moves between cells of different eNodeBs. In FIG. 1 LTE20 is shown comprising two MMEs, a first MME 31 represented by a solidline rectangle and indicated by dotted lines as communicating with UEsshown in the figures, and a second MME 32, shown in dashed lines, towhich first MME 31 may hand over UEs as the UEs move between cells.

Communication between two nodes in an LTE network is governed by aninterface that configures what data the nodes exchange, how packets thatcontain the data are formatted, and how the packets are transmittedbetween the nodes. Communication between different node pairs istypically governed by different interfaces. An interface that configurescommunications between two nodes is indicated in FIG. 1 by analphanumeric identifying the interface shown in an ellipse overlying adashed or solid line that extends between the two nodes. For example,eNodeBs in an LTE network exchange control signaling via an X2interface, and an X2 is shown in an ellipse overlying a dashed lineshown connecting the two eNodeBs in LTE network 20, and a UEcommunicates with an eNodeBs via a Uu interface shown in an ellipseoverlying a solid line between the UE and the eNode B. Similarly, asindicated in FIG. 1 , the MME node communicates control data with aneNodeB via an S1-MME interface. The S1-MME interface defines data to beexchanged and packet formatting in accordance with an S1-AP protocol andconfigures transmission of the data between the MME and the eNodeB usingthe SCTP (Stream Control Transmission Protocol) transport protocol. TheMME node communicates with the SGW node via an S11 interface, whichtunnels UDP packets between the nodes using a General Packet RadioService Tunneling Protocol (GTP) referred to as GTP-C. And, the HSScommunicates with the MME via an S6a interface that configures data inaccordance with a Diameter protocol and transmits Diameter messagesusing SCTP.

FIG. 2 schematically shows a Smart-vLB 100 coupled to LTE network 20operating to load balance processing of data comprised in controlpackets transmitted between MME node 31 and other nodes in the LTEnetwork, in accordance with an embodiment of the disclosure. By way ofexample, Smart-vLB 100 is assumed to comprise two, optionally virtual,non-Stateful load balancers, NS-vLBs, 101 and 102, and two, optionallyvirtual, Stateful load balancers, S-vLB 110 and S-vlB 111, and is shownload balancing packets it receives from LTE network 20 between,optionally three, vProbes 121, 122, and 123.

It is noted that whereas Smart-vLB 100 is schematically shown in FIG. 2and assumed to comprise virtual component entities, any or anycombination of component entities of a Smart-vLB in accordance with anembodiment, such a non-Stateful load balancer and/or a Stateful loadbalancer may comprise and/or have access to any of various physicaland/or virtual processors and memories that may be required to supportfunctionalities for which the component entities are responsible. Acomponent entity may be a physical or virtual entity or a combination ofphysical and virtual entities and may comprise or have access to amemory having any electronic and/or optical circuitry suitable forstoring data and/or computer executable instructions. The memory may, byway of example, comprise or have access to any one, or any combinationof more than one of a flash memory, random access memory (RAM), readonly memory (ROM), and/or erasable programmable read-only memory(EPROM). Similarly, a component entity may comprise and/or have accessto a processor having any electronic and/or optical processing circuitryconfigured to operate and provide functionalities that the componententity may require. The processor may by way of example comprise anyone, or any combination of more than one of, a standard x86 processor,microprocessor, an application specific circuit (ASIC), fieldprogrammable array (FPGA) and/or system on a chip (SOC).

Smart-vLB 100 is shown in FIG. 2 by way of example coupled to LTE 20 toreceive control packets mirrored from interfaces S1-MME, S3, and Gn,S11, Sv and SGs, S6a and S10 between MME 31, and eNodeBs, SGSN, SGW,MSC, HSS, and MME 32, respectively. Link controllers (not shown) in MME31 are configured to distribute substantially equal numbers of mirroredpackets from LTE network 20 to each non-Stateful vLB, NS-vLB 101 and102. Diameter packets mirrored to Smart-vLB 100 from interface S6a andGTP-C tunneled packets mirrored from interfaces, Sv, S3, S11 and S10 areselected as contextual packets. Packets mirrored to Smart-vLB 100 frominterfaces S1-MME and SGs are considered non-contextual packets. EachNS-vLB 101 and 102 forwards non-contextual packets that it receives insubstantially equal numbers to a given vProbe 121, 122, and 123 withoutreference to data in another packet. On the other hand, both NS-vLB 101and 102 forward Diameter packets that they receive to S-vLB 111, andGTP-C packets they receive to S-vLB 110. Each S-vLB load balancerforwards a packet it receives to a vProbe 121, 122, or 123 having anidentity which the S-vLB determines based on information from at leastone of a plurality of other packets that it receives.

For convenience of reference, S-vLBs 110 and 111 are labeled by thetypes of packets they respectively receive. S-vLB 110 is labeled “GTP-C”and S-vLB 111 is labeled “Diameter”.

FIG. 3 shows a flow diagram of an algorithm 200 by which NS-vLB 101 orNS-vLB 102, generically referred to as an NS-vLB, may determine where toforward packets they receive, in accordance with an embodiment of thedisclosure.

In a block 202 the NS-vLB receives and optionally time stamps a packetfrom the selected MME interfaces shown in FIG. 2 , and in a decisionblock 204 determines if the packet contains Application, Layer 7 of theOpen System Interconnection (OSI) model, data. If the packet does notcontain L7 data, in a block 205 the NS-vLB discards the packet andreturns to block 202. If on the other hand it is an L7 packet, NS-vLBmay proceed to a block 206 to determine if the packet is an S1APprotocol packet. If it is, NS-vLB optionally proceeds to a block 207 andextracts an eNode-B IP address of an eNodeB from which or to which thepacket was sent. Based on the extracted eNodeB IP address, NS-vLBcomputes, optionally in a block 208, an address of a destination vProbe,to which to forward the packet. By way of example, to compute a vProbedestination address, in block 208 NS-vLB may hash the extracted IPaddress, and use a look-up table (LUT) that maps hashes of IP addressesof eNodeBs from which MME 31 may receive a packet to determine to whichvProbe, 121, 122, or 123 to forward the packet. In a block 209 NS-vLBmay encapsulate the packet for tunneling to the determined destinationvProbe address, and in a block 210 may transmit the packet to thedestination vProbe. Any suitable tunneling protocol such as the GRE(General Routing Encapsulation) tunneling protocol may be used in block210 to tunnel the packet to the destination vProbe. After transmittingthe S1AP packet to its destination vProbe, NS-vLB returns to block 202to receive another packet from the selected MME interfaces.

If a packet received by NS-vLB is found to be an L7 packet in decisionblock 204, but not an S1AP packet in decision block 206, NS-vLB mayproceed to determine in a decision block 212 if the received packet isan SGs protocol packet. If the packet is an SGs packet, optionally in ablock 213, NS-vLB extracts an MSC IP address from the packet.Optionally, in blocks 208, 209, 210, NS-vLB proceeds to determine an IPaddress of a destination vProbe 121, 122, or 123, to which to forwardthe packet based on the extracted MSC IP address, and tunnel the packetto the destination vProbe. Thereafter, NS-vLB may return to block 202 toreceive another packet.

If a received packet is determined to be an L7 packet but not an S1AP orSGs packet, NS-vLB may proceed from decision block 212 to a decisionblock 214 to determine if the packet is a GTP-C packet. If it is a GTP-Cpacket, NS-vLB may proceed to blocks 215 and 216 to tunnel the packet toGTP-C stateful load balancer S-vLB 110 (FIG. 2 ). Following transmissionof the packet to GTP-C S-vLB 110, NS-vLB may return to block 202 toreceive another packet. If on the other hand in decision block 214NS-vLB finds that the packet is not a GTP-C packet, NS-vLB may proceedto a block 217 to determine if the packet is a Diameter packet. If it isa Diameter packet, NS-vLB may proceed to blocks 218 and 219 to tunnelthe packet to Diameter stateful load balancer S-vLB 111 (FIG. 2 ), andthereafter return to block 202.

If on the other hand, in decision block 217 NS-vLB finds that the packetis not a Diameter packet, NS-vLB may proceed to blocks 220 and 221 totunnel the packet to a default vProbe or vProbe group, which may be anyone of vProbes 121, 122, or 123 and thereafter return to block 202.

FIGS. 4A-4B show a flow diagram of an algorithm 300 by which,optionally, Diameter stateful virtual load balancer S-vLB 111 (FIG. 2 )processes Diameter packets it receives from non-stateful NS-vLB loadbalancers 101 and 102.

In a block 302 Diameter stateful load balancer S-vLB 111 receives anSCTP (Stream Control Message Protocol) packet comprising one or morecomplete and/or fragmented Diameter messages. In a decision block 304load balancer S-vLB 111 determines if the SCTP packet comprisesfragmented Diameter messages or a single, complete, non-fragmented,Diameter message. If the packet is a “fragmented packet”, comprisingfragmented Diameter messages, load balancer S-vLB 111 proceeds to ablock 306 and reassembles a Diameter message, a fragment of which iscomprised in the SCTP packet. Reassembly comprises aggregating asufficient number of SCTP packets to provide load balancer S-vLB 111with a sufficient number of fragments to enable reassembly of a completeDiameter message. Following reassembly, load balancer 111 may proceed toa decision block 308. If in decision block 304, load balancer S-vLB 111determines that the SCTP packet received in block 302 is not fragmented,load balancer S-vLB 111 may proceed directly to decision block 308.

In decision block 308 load balancer S-vLB 111 optionally determines ifthe Diameter message is an Authentication Information Request message,and if it is, may proceed to a block 310 in which the load balancerextracts an IMSI (International Mobile Subscriber Identity) and ahop-to-hop identifier (H2H ID), source and destination IP addresses thatrespectively identify a UE associated with the Diameter message and asession in which the UE is engaged. In a block 312 load balancer 111stores the IMSI, H2H ID, and source and destination addresses in atransaction table that the load balancer maintains in a memory.Thereafter the load balancer may proceed to a block 328 in which, basedon the extracted H2H ID it determines an IP address of a vProbe 121,122, or 123, to which the load balancer will forward the packet orpackets containing the Diameter message for processing. In a block 330load balancer S-vLB 111 encapsulates the Diameter message packet orpackets for tunneling to the determined IP address, and in a block 332transmits the Diameter message to the vProbe IP address.

In a block 334, load balancer S-vLB 111 determines if the transmittedDiameter message is a last Diameter message in the SCTP packet. If itis, load balancer S-vLB 111 proceeds to a junction “X” and returns toblock 302 to receive and process another SCTP packet. On the other hand,if it is not a last Diameter message, load balancer S-vLB 111 mayproceed to a junction “Y” and returns to block 304 to determine if thereis another Diameter message in the packet for processing and if it isfragmented.

In processing a Diameter message, if in decision block 308 load balancerS-vLB 111 determines that the Diameter message is not an AuthenticationInformation Request message, the load balancer may proceed to a decisionblock 314 to determine if the Diameter message is a response to anAuthentication Information Request message. If the message is not aresponse to an Authentication Information Request message load balancerS-vLB 111 may proceed from decision block 314 to a block 326 to extracta H2H ID from the message and in blocks 328-332 to transmit the messageto a vProbe having an IP address determined from the H2H ID. In block334 the load balancer determines whether to return to block 302 andreceive another SCTP packet or to block 304 to process another Diametermessage comprised in the current SCTP packet.

If in block 314 load balancer S-vLB 111 determines that the Diametermessage is an Authentication Information Answer message the loadbalancer may proceed to a block 316 and extract an H2H ID from themessage. In a block 318 the load balancer may attempt to match theextracted H2H ID, source and destination IP addresses with an H2H ID,source and destination IP addresses message previously logged into thetransaction table to identify a Diameter Authentication InformationRequest message with which the Authentication Information Answer messageis associated. In a decision block 320, if the matching attempt issuccessful, Diameter load balancer S-vLB 111 proceeds to a block 323 andgenerates a data record comprising the authentication vector extractedin block 316 from the Diameter Authentication Information Answer messageand the IMSI extracted from the associated Diameter AuthenticationInformation Request message. The combination of an encryption keycomprised in the authentication vector and the IMSI enable a vProbe 121,122, or 123 to decipher a packet that it receives which is associatedwith the session associated with the IMSI. In a block 324 load balancerS-vLB 111 may transmit the data record to each of vProbes 121, 122, and123.

Optionally, following transmission of the data record, load balancerS-vLB 111 may proceed to blocks 328-332 to determine a destination IPaddress of a vProbe 121, 122, or 123 to which the DiameterAuthentication Information Answer message should be sent. Thedestination IP address determined for the Authentication InformationAnswer message is based on the H2H ID associated with the AuthenticationInformation Answer message, which H2H ID is shared with theAuthentication Information Request message matched to the AuthenticationInformation Answer message in block 318. As a result, the vProbe IPaddress determined for the Diameter Authentication Information Answermessage is the same as the vProbe IP address determined for theassociated Diameter Authentication Information Request message. And inblock 332, after encapsulation in block 330, the Diameter AuthenticationInformation Answer message is transmitted to the same vProbe foranalysis to which the associated Diameter Authentication InformationRequest message was sent. After transmission of the DiameterAuthentication Information Answer, message load balancer 111 may proceedto block 334 to determine whether to return to block 302 and receiveanother SCTP packet or to block 304 to process the next Diameter messagein the current SCTP packet. It is noted that whereas in the abovedescription load balancer S-vLB 111 proceeds to blocks 328-332 afterhaving generated and transmitted an associated data record, in anembodiment load balancer S-vLB 111 may generate and transmit the datarecord after having proceeded to execute, or during execution of, blocks328-332.

In a case for which in decision block 320 load balancer S-vLB 111determines that matching in block 318 has failed, the load balancer mayproceed to perform a “Wait and See” procedure. In the Wait and Seeprocedure load balancer S-vLB 111 stores the Diameter AuthenticationInformation Answer message in a holding memory, optionally a RAM (randomaccess memory). In the holding memory the Diameter message may be storedwith other messages in order of their respective time stamps provided bythe non-Stateful load balancer NS-vLB 101 or NS-vLB 102 from which themessage was forwarded to load balancer S-vLB 111. In an embodiment, theload balancer S-vLB 111 processes the message in order of its time stampfollowing a predetermined from a time at which the message was stored inthe holding memory to make at least one additional attempt at matchingthe Authentication Information Answer message with an AuthenticationInformation Request message. If in a decision block 322 the at least oneadditional attempt is successful, load balancer S-vLB 111 may proceed toblocks 323 and 324 to generate and transmit an IMSI plus authenticationvector data record to all vProbes 121, 122, and 123. If however, indecision block 322 load balancer 111 determines that the at least oneadditional attempt has failed, the load balancer may proceed to executeblocks 328-334.

In an embodiment, to reduce a probability that an AuthenticationInformation Answer message reaches Diameter load balancer S-vLB 111before the associated Authentication Information Request message reachesthe load balancer, the load balancer may store all Diameter messagesthat it receives in a holding memory prior to attempting to matchmessages. The load balancer holds the messages in the holding memory fora given delay time before processing to reduce the probability that theload balancer will not find a match for an Authentication InformationAnswer

FIGS. 5A-5C show a flow diagram of an algorithm 400 by which,optionally, GTP-C, stateful virtual load balancer S-vLB 110 (FIG. 2 )processes GTP-C packets it receives from non-stateful NS-vLB loadbalancers 101 and 102.

In a block 402 load balancer S-vLB 110 receives a GTP-C message packet,and optionally in a block 410 determines if the message is a createsession request. If it is, in a block 411 the load balancer extracts asequence number, an IMSI, and a GTP-C MME F-TEID (Fully Qualified TunnelEndpoint Identifier) located in the packet payload. In a block 412 loadbalancer S-vLB 110 may store the MME F-TEID and IMSI in a session tablethat the load balancer maintains, and in a block 413 stores the packetin a holding memory, optionally a RAM. From block 413 the load balancermay return to block 402 to receive another packet.

If in decision block 410 load balancer S-vLB 110 determines that themessage is not a create session message, the load balancer may proceedto a decision block 420 to determine if the message is a create sessionresponse message. If it is, load balancer S-vLB 110, optionally in ablock 421, extracts a sequence number and a GTP-C SGW F-TEID andoptionally proceeds to find a match for the extracted sequence numberwith a previously extracted create session request sequence numberstored in the session table. Load balancer S-vLB 110 may store the GTP-CSGW F-TEID in the session table and optionally proceed to a block 471.In block 471 based on the GTP-C MME F-TEID extracted from the associatedcreate session request and stored in the session table in block 412,load balancer 110 determines an IP address for a vProbe 121, 122, or 123to which to send for processing the session response packet and itsassociated session request packet stored in block 413 in RAM. In a block472, the load balancer encapsulates the packets for tunneling, and in ablock 473 transmits the encapsulated packets to the determinedIP-address of the destination vProbe. Load balancer S-vLB 110 may thenreturn to block 402 to receive another GTP-C packet.

If in decision block 420 load balancer S-vLB 110 determines that theGTP-C message is not a create session response, the load balancer mayproceed to a decision block 430 to determine if the message is a modifybearer request message. If it is, load balancer S-vLB 110 may proceed toa block 431 in which the load balancer extracts a GTP-C F-TEID an S1-UeNodeB F-TEID, and an eNodeB S1-U IP address from the packet. In a block432 load balancer 110 matches the extracted GTP-C TEID with a GTP-C SGWF-TEID previously stored in the load balancer session table for a createsession response and stores the S1-U eNodeB F-TEID and eNodeB S1-U IPaddress in the session table so that it may be associated with thecreate session response of the same GTP-C SGW F-TEID. In a block 434 theload balancer stores the packet in the holding memory RAM and may returnto block 402 to receive another packet.

If in decision block 430 load balancer S-vLB 110 determines that thepacket is not a modify bearer request, the load balancer may continue toa decision block 440 to determine if the message is a modify bearerresponse message. If it is a modify bearer response message, the loadbalancer may proceed to a block 441 to extract a GTP-C F-TEID from thepacket as well as an S1-U SGW F-TEID, and in a block 442 match theextracted GTP-C F-TEID with a GTP-C MME F-TEID previously stored in thesession table. In a block 443 load balancer S-vLB 110 stores theextracted S1-U SGW F-TEID in the session table so that it may beassociated with the other data in the session table having the sameGTP-C MME-TEID. In a block 444 load balancer S-vLB 110 generates a datarecord comprising the IMSI, S1-U eNodeB F-TEID, and the S1-U SGW F-TEIDthat are stored in the session table and associated with the same GTP-CMME F-TEID. In a block 445 the load balancer may determine a vProbe IPaddress based on the eNodeB S1-U IP address extracted in block 431 forthe modify bearer request associated with the modify bearer response.And in a block 446 the load balancer may transmit the data record to thevProbe 121, 122, or 123 having the determined vProbe IP address. Loadbalancer S-vLB 110 may then proceed to blocks 471-473 to transmit theassociated modify bearer request and modify bearer response packets forprocessing to a vProbe having an IP address determined from the GTP-CMME F-TEID that is associated with the modify session request andresponse packets. The vProbe to which the associated modify bearerrequest and modify bearer response packets are sent is also the samevProbe to which the create session request and response packetsassociated with the same GTP-C MME F-TEID were transmitted. From block473 the load balancer may return to block 402 to receive another packet.

In decision block 440 if load balancer S-vLB 110 determines that thepacket is not a modify bearer response, the load balancer may proceed toa block 450 to determine if the packet is another type of message packetdifferent from a create session or modify bearer request. If it is, in ablock 451 the load balancer extracts a GTP-C TEID from the packet and ina block 452 matches the extracted GTP-C TEID with a GTP-C MME TEID or aGTP-C SGW F-TEID previously stored in the session table. Load balancerS-vLB 110 may then proceed to blocks 471-473 to transmit the packet to avProbe having an IP address determined from the GTP-C MME F-TEIDassociated with the session to which the create session request andresponse packets associated with the same GTP-C MME F-TEID belong. Theload balancer may then proceed to block 402 to receive another packet.

It is noted that whereas a Wait and See procedure is described withrespect to load balancer S-vLB 111 it may also be used where necessaryby load balancer S-vLB 110.

The above discussion describes a Smart-vLB system balancing dataprocessing for packets transmitted via MME interfaces in accordance withan embodiment of the disclosure. However, a Smart-vLB in accordance withan embodiment is not limited to load balancing for packets transmittedvia MME interfaces, and may be configured to load balance forcommunication interfaces via which entities of a network other than anMME communicate. By way of example, FIG. 6 schematically shows aSmart-vLB 100 coupled to LTE network 20 operating to load balanceprocessing of data comprised in control plane and user plane packetstransmitted between the SGW node, the PGW node, and other nodes in theLTE network, in accordance with an embodiment of the disclosure. In thefigure Smart-vLB 100 is shown by way of example coupled to LTE 20 toreceive and load balance packets mirrored from interfaces S1-U, S4, S11,S5/S8, SGi, Gn, Gi, Gx. A Smart-vLB may simultaneously be coupled tointerfaces of more than one network entity and be configured tosubstantially simultaneously be responsible for load balancingcommunications for all the more than one network entity.

FIG. 7A shows a flow diagram of an algorithm 500 by which NS-vLB 101 orNS-vLB 102, may determine where to forward packets they receive from anSGW or PGW interface to which Smart-vLB 100 is coupled, in accordancewith an embodiment of the disclosure.

In a block 502 the NS-vLB (NS-vLB 101 or NS-vLB 102) receives andoptionally time stamps a packet from the selected SGW interfaces shownin FIG. 6 , and in a decision block 504 determines if the packetcontains a GTP-C protocol packet. If it is a GTP-C packet, NS-vLB mayproceed to blocks 506 and 508 to tunnel the packet to GTP-C statefulload balancer S-vLB 110 (FIG. 6 ). Following transmission of the packetto S-vLB 110, NS-vLB may return to block 502 to receive another packet.If on the other hand in decision block 504 NS-vLB finds that the packetis not a GTP-C packet, NS-vLB may proceed to a block 510 to determine ifthe packet is a Diameter packet. If it is a Diameter packet, NS-vLB mayproceed to blocks 512 and 514 to tunnel the packet to Diameter statefulload balancer S-vLB 111 (FIG. 6 ), and thereafter return to block 502.

If on the other hand, in decision block 510 NS-vLB finds that the packetis not a Diameter packet, NS-vLB may proceed to decision block 516 todetermine if the packet is a GTP-U, Gi, or SGi packet and if not,discards the packet or transmits the packet to a default vProbe, andreturns to block 502 to receive another packet. On the other hand, if indecision block 510, NS-vLB determines that it is a GTP-U, Gu, or SGipacket, NS-vLB may proceed to a block 518. In block 518 NS-vLB mayextract an MS IP (Mobile Station IP) address from the packet and in ablock 520 the NS-vLB may determine an address for a destination vProbe,vProbe 121, 122, or 123, to which to transmit the packet for processing.In a block 522 NS-vLB encapsulates the packet for tunneling andtransmits the encapsulated packet to the determined address of thedestination vProbe. Following transmission NS-vLB may return to block502 to receive another packet.

FIG. 7B shows a flow diagram of an algorithm 600 by which, optionally,GTP-C, stateful virtual load balancer S-vLB 110 (FIG. 6 ) processes aGTP-C packet mirrored from a SGW interface to which Smart-vLB 100 iscoupled that S-vLB 110 receives from non-stateful NS-vLB load balancer101 or 102.

In a block 602 load balancer S-vLB 110 receives a GTP-C message packet,and optionally in a block 610 determines if the message is a createsession request. If it is, in a block 611 the load balancer extracts asequence number, an IMSI, and a GTP-C MME F-TEID (Fully Qualified TunnelEndpoint Identifier) located in the packet. In a block 612 load balancerS-vLB 110 may store the sequence number, MME F-TEID, and IMSI in asession table that the load balancer maintains, and in a block 613stores the packet in a holding memory, optionally a RAM. From block 613the load balancer may return to block 602 to receive another packet.

If in decision block 610 load balancer S-vLB 110 determines that themessage is not a create session message, the load balancer may proceedto a decision block 620 to determine if the message is a create sessionresponse message. If it is, GTP-C virtual load balancer S-vLB 110,optionally in a block 621, extracts a sequence number, a GTP-C SGWF-TEID and a MS IP address. Thereafter in a block 622, S-vLB 110optionally proceeds to find a match for the extracted sequence numberwith a previously extracted create session request sequence numberstored in the session table. In a block 623 load balancer S-vLB 110 maystore the GTP-C SGW F-TEID in the session table and, optionally, proceedto a block 650. In block 650 based on the MS IP address extracted fromthe associated create session request and stored in the session table inblock 623, load balancer S-vLB 110 determines an IP address for adestination vProbe 121, 122, or 123 to which to send for processing, thesession response packet and its associated session request packet storedin RAM in block 613. In a block 652, the load balancer may encapsulatethe packets for tunneling and transmit the encapsulated packets to thedetermined IP-address of the destination vProbe. Load balancer S-vLB 110may then return to block 602 to receive another GTP-C packet.

If in decision block 420 load balancer S-vLB 110 determines that theGTP-C message is not a create session response, the load balancer mayproceed to a decision block 630 to extract a GTP-C F-TEID from thepacket. In a block 632 load balancer S-vLB 110 matches the extractedGTP-C TEID with a GTP-C MME F-TEID or GTP-C SGW F-TEID previously storedin the load balancer session table and may proceed to block 650. Inblock 650 S-vLB 110 calculates an IP address for a destination vProbe121, 122, or 123 to which to send the packet for processing based on theMS IP address stored in the session table for the GTP-C MME F-TEID orGTP-C SGW F-TEID matched with the GTP-TEID in block 632. In a block 652,S-vLB 110 encapsulates the packet and transmits the packet to thedetermined IP address of the destination vProbe and may return to block602 to receive another packet.

In an embodiment Smart-vLB 100 may be configured as described below toload balance user packets in packet flows of EPS sessions between UEsand PDNs connected by LTE 20 and control, “throttle”, transmission ratesof copies of user packets to a probe for processing to preventoversubscribing probe capacity. PDNs to which an LTE may connect areidentified by PDN-IDs, which are conventionally also referred to asAccess Point Names (APNs), and EPS sessions between UEs and PDNs mayalso be referred to as PDN connections.

To perform the load balancing, Smart-vLB 100 may, as schematically shownin FIG. 6 , be configured to receive and process control packetsmirrored to Smart-vLB 100 from control plane interfaces, optionally, asdiscussed above, as well as user packets from user plane interfaces,such as optionally S1-U, S5/S8, and/or SGi interfaces. S1-U and S5/S8interfaces provide bearers that are concatenated with a data radiobearer (DRB) to provide EPS bearers through which user packets in an EPSsession are tunneled from an eNodeB in communication with LTE 20 towhich a UE is connected and a PDN to which LTE 20 connects the UE.Interface S1-U governs user packet configuration and transportationthrough an S1 bearer that provides the EPS bearers with a GTP-U tunnelto transport user packets in the EPS session between the eNodeB to whichthe UE is connected and a serving gateway SGW of the LTE. InterfaceS5/S8 governs user packet configuration and transportation through anS5/S8 bearer which provides a GTP-U tunnel to transport user packets inthe EPS session between the SGW and a Packet Data Network Gateway (PGW)which connects LTE 20, and thereby the UE, to the PDN. The SGi interfaceconfigures communications between the PGW and the PDN.

In accordance with an embodiment, Smart-vLB 100 load balances andthrottles copies of user plane packets from UEs connected to LTE 20based on a particular feature, a target feature, that characterizes thepackets. As noted above, the target feature of the packets may by way ofexample, be an IMSI or an IMEI (International Mobile Equipment Identity)of a UE associated with transmission of the packets, an MSISDN (MobileStation International Subscriber Directory Number), APN (Access PointName), QCI, (Quality of Service Class Identifier), and/or a CGI (CellGlobal Identifier). For convenience of presentation, it is assumed thatthe target feature of the packets is their IMSI. FIG. 8 exhibits a flowdiagram 700 by which Smart-vLB 100 may associate user packets propagatedin EPS sessions through LTE 20 with IMSIs of the UEs participating inthe sessions, and based, optionally, on the IMSIs load balance the userpackets.

IMSIs of the UEs that the Smart-vLB associates with the user packets.FIG. 8 exhibits a flow diagram 700 by which Smart-vLB 100 may associateuser packets propagated in EPS sessions through LTE 20 with IMSIs of theUEs participating in the sessions, and based, optionally, on the IMSIsload balance the user packets.

In a block 702 Smart-vLB 100 may initiate at least one holding bufferfor each NS-vLB 101 and 102, in which the NS-vLB temporarily stores userplane packets it receives for which the NS-vLB cannot, or is not asdiscussed below, fully configured to associate an IMSI.

In a block 704 non-stateful NS-vLBs, by way of example NS-vLBs 101 and102, comprised in Smart-vLB 100 receive control and user plane packetsfrom interfaces that the Smart-vLB monitors for load balancing. In ablock 706 the NS-vLBs forward control plane packets for processing tostateful S-vLBs comprised in Smart-vLB 100, by way of example GTP-CS-vLB 110 and/or Diameter S-vLB 111. Optionally, in a block 708 theS-vLBs process control plane packets which they receive that areinvolved in establishing, modifying, tearing down, or configuring an EPSsession to acquire, in accordance with an embodiment of the disclosure,an ensemble, hereinafter also referred to as a characterizing vector, ofcharacterizing variables. A characterizing variable of an EPS session isa variable that may be used by itself or in conjunction with anothercharacterizing variable, to associate a user packet transported in theEPS session with an IMSI, which may also be referred to as a sessionIMSI, of a UE participating in the EPS session. A characterizingvariable may also be referred to as a component of a characterizingvector to which the characterizing variable belongs. The characterizingvariables may comprise various transport path variables, such as userand NE IDs, and/or user subscription variables. By way of example thecharacterizing variables may comprise, at least one, or any combinationof more than one, of a GUTI and/or IP address assigned by an MME in LTE20 to the UE for use in the EPS session, the APN of a PDN to which theLTE connects the UE for the session, at least one user plane TEID thatidentifies a tunnel end of a bearer comprised in the session EPS bearer,and/or an IP address of an NE in LTE 20 that supports the session.Characterizing variables that are associated with an IMSI of a sessionmay be determined by S-vLB 110 and/or S-vLB 111 similarly to the mannerby which Smart-vLB 100 associates IMSIs with data items in data and/orsession records stored in a transaction or session table maintained bySmart-vLB 100 and described with reference to algorithms 300, 400,and/or 500.

In an embodiment, the various S-vLBs that receive and process controlplane packets to identify components for a characterizing vector of asession may store the components that the S-vLB identifies in a sessionrecord in a session table maintained in a memory of the Smart-vLB. Whenthe session record comprises a number and variety of characterizingvariables sufficient to enable substantially any user packet propagatedvia LTE network 20 in the session to be associated with the sessionIMSI, the characterizing vector may be considered complete. Acharacterizing vector for an EPS session that connects a UE with a PDNmay be defined to have a predetermined number and type of components,such as by way of example components selected from the components notedabove, and when a value is assigned to each component, thecharacterizing vector may be considered to be complete. Optionally, acharacterizing vector is complete when Smart-vLB 100 associates a lastTEID that is used in transporting packets via the EPS session with thesession IMSI.

In a block 710 Smart-vLB 100 may determine that a characterizing vectorfor a given EPS session is complete and transmit to each NS-vLB 101, 102or a selection of NS-vLBs in the Smart-Vlb, the complete characterizingvector for use in associating user packets with the EPS session IMSI andload balancing the associated packets. In an embodiment, S-vLBs inSmart-vLB may be configured to communicate with each other and/or thesession table, to determine when a characterizing vector is complete. Toprovide each of the NS-vLBs with a complete characterizing vector theS-vLBs may operate to assign at least one of the S-vLBs to transmit acomplete characterizing vector to at least one of the NS-vLBs. In anembodiment, to provide Smart-vLB 100 with advantageously small latencyfor processing user packets, an S-vLB that operates to identifycharacterizing vector components may be configured to transmit thecomponents and/or the vector that it identifies to an NS-vLB using arelatively high-speed communications the high-speed protocol providestransit times for transmission of the components and/or the vectorbetween the S-vLB and NS-vLB that is less than 10 ms. Optionally, thecommunications protocol provides transit times less than about 5 ms.Optionally, the communications protocol provides transit times less thanabout 2 ms. In an embodiment the S-vLB high speed protocol is a Zer0MQprotocol.

In the above description, Smart-vLB 100 appears to be configured inaccordance with an embodiment to operate a centralized session table forcompilation and storage of characterizing vectors and to distribute acharacterizing vector to NS-vLBs only after values have been providedfor all of the vector components and the vector is considered to becomplete. However, practice of an embodiment of the disclosure is notlimited to transmitting complete characterizing vectors to NS-vLBs. Inan embodiment S-vLBs may be configured, to transmit characterizingcomponents for an EPS session, “piecemeal” substantially as they areidentified, to an NS-vLBs in Smart-vLB 100 that is configured to receivethe characterizing components piecemeal and compile the components intoa characterizing vector. For example, in response to identifying aparticular characterizing component of an EPS session, and optionallyindependent of identifying and/or transmitting another characterizingcomponent, an S-vLB may transmit a copy of the particular component toeach NS-vLB together with information sufficient for identifying an IMSIwith which the particular component is associated. The NS-vLB may storethe particular component it receives together with other components of acharacterizing vector associated with the same IMSI with which theparticular component is associated in a same record that the NS-vLBmaintains and uses to compile a characterizing vector.

Upon receiving and/or compiling a complete characterizing vector, anNS-vLB is fully configured to associate user packets in the given EPSsession with the session IMSI and to load balance the packets, andoptionally in a block 712, the NS-vLB begins using the characterizingvector to associate user packets in the NS-vLB buffer and user packetsas they are received by the NS-vLB, optionally without first storing thepackets in the buffer. Using the characterizing vector to associate auser packet with the given IMSI may comprise determining if a header ofthe user packet contains a data item that matches a component of thecharacterizing vector. For example, the NS-vLB may associate the userpacket with the given IMSI because the user packet header may comprise aTEID which the NS-vLB determines matches a component of thecharacterizing vector that is a TEID of an EPS bearer associated withthe given IMSI. Or the NS-vLB may find that the user packet header maycomprise an IP address and APN that matches components of thecharacterizing vector.

In a block 714 the NS-vLB may apply a selection filter to select fromamong user packets the NS-vLB has associated with an IMSI, a subset ofuser packets for load balancing and transmitting to v-Probes 121-123 forprocessing and to drop from transmitting to the v-Probes user packetsthat are not selected. In an embodiment, the NS-vLB forwards selecteduser packets associated with a same IMSI to a same vProbe 121-123 forprocessing. It is noted that in an embodiment the NS-vLB may operateproperly to forward selected user packets associated with a same IMSI toa same vProbe 121-123 for processing, independent of whether the UEassociated with the IMSI is assigned an IPv4, IPV6, or Dual Stack MS-IPaddress, and/or is assigned more than one MS-IP address.

In an embodiment of the disclosure Smart-vLB 100 comprises a library ofone or more selection filters that the NS-vLB may be configured to applyto select user packets for load balancing. Optionally, the selectionfilter library comprises a “depletion filter” that operates to controlor throttle a number of packets that Smart-vLB transmits to vProbes121-123 for processing. In an embodiment, the depletion filter operatesto select packets for load balancing and forwarding to vProbes 121-123,to limit a number of packets that Smart-vLB 100 transmits per unit time,also referred to as an LB transmission rate, to a given probe of vProbes121-123 or globally to all the vProbes for processing. Selection ofpackets to limit an LB transmission rate may be a function of at leastone LB transmission rate upper limit and an LB transmission rate ofSmart-vLB 100. For example, if the LB transmission rate of Smart-vLB 100is less than the at least one LB transmission rate upper limit, thedepletion filter may operate so that all user packets that the NS-vLBsassociate with an IMSI may be transmitted to a vProbe 121-123. On theother hand, the depletion filter may operate to select only a subset ofuser packets for transmission to vProbes 121-123 if in the absence ofselection, the LB transmission rate would exceed the transmission rateupper limit.

Optionally, the at least one LB transmission rate upper limit is basedon a capacity of at least one of vProbes 121-123 to process userpackets. In an embodiment, the upper limit is equal to a maximum packetprocessing rate of vProbes 121-123 or a fraction less than one of themaximum packet processing rate. The fraction less than one may, by wayof example be less than or equal to about 0.90. Optionally the fractionless than one is less than or equal to about 0.80. The upper limit maybe a function of time of day and/or a geographical region of LTE 20 forwhich the upper limit is to be applied. In an embodiment the upper limitis a dynamic limit continuously determined in accordance with aproportional-integral-derivative (PID) algorithm to maintain the loadbalancing rate at a predetermined desired rate.

In an embodiment, the depletion filter is configured to select userpackets received by NS-vLBs 121-123 for transmission based on the IMSIswith which the packets are associated so that only packets associatedwith an IMSI that is a member of a group of selected IMSIs are loadbalanced and forwarded to vProbes for processing. Optionally memberIMSIs are randomly selected.

Optionally, to implement selection of packets based on their respectiveIMSIs, Smart-vLB 100 may comprise at least one whitelist and at leastone blacklist of IMSIs. IMSIs listed in the at least one whitelist maybe selected for transmission to vProbes for processing and IMSIs listedin the at least one blacklist may be dropped. In an embodiment the atleast one whitelist and/or at least one blacklist may be dynamicallyupdated as a magnitude of packet traffic supported and/or demandedchanges. For example, a number of whitelisted IMSIs may be decreased asmagnitude of traffic increase and/or new IMSIs may be added or removedto the blacklist and/or whitelist as users connect to and/or disconnectfrom LTE 20.

In an embodiment a depletion filter, also referred to as a packet flowdepletion filter may operate to select user packets for forwarding tovProbes based on the respective packet flows to which the packetsbelong. Packets belonging to a same packet flow in LTE 20 are identifiedby a same 5-tuple of information elements that includes a source IP, adestination IP, a source port number, a destination port number, and acommunications protocol. A packet flow depletion filter that NS-vLBs inSmart-vLB 100 may apply to packets that Smart-vLB 100 receives from LTE20 operates to select packets for load balancing from a portion of thepacket flows in LTE 20 from which Smart-vLB 100 receives packets. TheNS-vLBs forward packets selected by the packet flow depletion filterthat belong to a same packet flow to a same vProbe.

In an embodiment the packet flow depletion filter is configured tooperate based on a set of operating parameters optionally comprising anactivation packet traffic threshold, activation delay, deactivationpacket traffic threshold, deactivation delay, and dynamic depletionfactor. The packet flow depletion filter is configured to activate todrop packets belonging to a fraction of the packet flows extant in LTE20 following an activation delay from a time at which packet trafficthat Smart-vLB 100 receives from LTE 20 exceeds the activation packettraffic threshold. The fraction of the packet flows from which theNS-vLBs drop packets may be substantially equal to the dynamic depletionfactor. The NS-vLBs determine the dynamic depletion factor as a functionof magnitude of packet traffic that Smart-vLB 100 receives from LTE 20so that a transmission rate of packets that Smart-vLB 100 forwards tovProbes 121-123 for processing does not exceed capacity, that isoptionally a maximum processing rate, of the vProbes to process thepackets. The packet flow depletion filter is configured to deactivateand cease dropping packets following a deactivation delay from a time atwhich packet traffic that Smart-vLB 100 receives from LTE 20 drops belowthe deactivation activation packet traffic threshold. Optionally, theactivation and deactivation packet traffic thresholds are substantiallythe same. In an embodiment the activation and deactivation packettraffic thresholds are different. Optionally, the activation anddeactivation delays are substantially the same. In an embodiment theactivation and deactivation delays are different.

It is noted that Smart-vLB 100 may be configured to apply a packet flowdepletion filter to throttle packet transmission to vProbes 121-123without having recourse to associating packets with an IMSI or aparameter other than an IMSI.

In an embodiment, the filter library comprises a “discrimination filter”that may be applied to select a portion of packets that Smart-vLB 100receives for load balancing based on features of user subscriberagreements under which users access and transmits packets over LTE 20. Adiscrimination filter may be operable to select packets for transmissionand processing by vProbes 121-123 based on features such as QoS, dataservices, and/or roaming policies of subscriber agreements associatedwith IMSIs. By way of example, NS-vLBs 101 and 102 may select packets asa function of QoS associated with an IMSI subscriber policy so that onlypackets associated with a QoS greater than a threshold QoS are selectedfor load balancing and forwarding to vProbes 121-123.

In an embodiment Smart-vLB 100 may use more than one selection filter toselect packets that the Smart-vLB transmits to a vProbe 121-123. Forexample, Smart-vLB 100 may use a depletion filter together with adiscrimination filter to select and limit a number of packets thatSmart-vLB load balances and forwards to vProbes 121-123. Thediscrimination filter may be used to select the packets based on a QoSthreshold determined responsive to a LB transmission rate upper limitdetermined by a depletion filter. In an embodiment the QoS threshold mayrise as a number of user packets that Smart-vLB 100 receives from LTE 20for load balancing increases.

It is noted that whereas in the above description packets were selectedfor load balancing and/or throttling based on their association with anIMSI, packets may be selected for load balancing and/or throttling inaccordance with an embodiment of the disclosure based on a parameter orparameters other than an IMSI. For example, packets may be selectedand/or throttled based on an IMEI, an MSISDN, GUTI, APN, QCI, and/or aCGI with which they may be associated. It is also noted that whereasSmart-vLB 100 is schematically shown and described as load balancing fora 4G LTE communications network a Smart-vLB in accordance with anembodiment may operate to load balance and throttle for 5G architectureand other networks configured to implement control and user planeseparation (CUPS). A Smart-vLB in accordance with an embodiment may beadvantageous for load balancing CUPS configured communications networksbecause the Smart-vLB architecture may use entities that operate asstateful load balancers, which are different from entities that operateas non-stateful load balancers. For example, the functionalities ofNS-vLBs in Smart-vLB 100 may be provided by entities that are differentfrom entities that provide the functionalities of S-vLBs 110-111. As aresult, Smart-vLB 100 may provide advantageous processing efficienciesand scaling flexibilities by coupling NS-vLBs 101-102 to receive andprocess user plane packets in a CUPS configured communications networkand S-vLBs 110-111 to receive and process control plane packets from theCUPS network.

It is also noted that whereas Smart-vLB 100 is described as initiatingload balancing and/or throttling only after receiving a completecharacterizing vector, in an embodiment a Smart-vLB may operate to loadbalance using characterizing components of a characterizing vector asthe Smart-vLB receives them, without waiting to receive all thecharacterizing components of the characterizing vector.

As noted above a Smart-vLB in accordance with an embodiment of thedisclosure may be configured to SIT throttle and/or SILBA load balance,control plane and/or user plane packets propagated over a communicationsnetwork. FIG. 9 shows a flow diagram 800 of a procedure by whichSmart-vLB 100 may SIT throttle and SILBA load balance user plane packetspropagated over communications network 20, in accordance with anembodiment of the disclosure.

In a block 802 of procedure 800 an NS-vLB, such as NS-vLB 101 or 102 ofSmart-vLB 100 (FIG. 6 ) receives a packet from an interface ofcommunications network 20 to which the Smart-vLB is connected.Optionally in a decision block 804 the NS-vLB determines whether or notthe received packet is a control plane packet. If the received packet isa control plane packet, such as a GTP-C packet, optionally in a block806 the NS-vLB transmits the packet to an S-vLB, such as S-vLB 110, andin a block 807 the S-vLB load balances the packet to a relevant vProbe.In a block 808 the S-vLB determines if the control plane packet belongsto a Create Session Request/Response, Delete Session Request/Response,or Modify Bearer Request/Response transaction for a user session. If itis not, in a block 809 S-vLB 110 optionally discards the packet andwaits to receive another control plane packet from an NS-vLB. On theother hand, if the received control plane packet is a Create SessionRequest/Response, Delete Session Request/Response, or Modify BearerRequest/Response user session packet, optionally in a block 810, theS-vLB processes the packet to extract at least one session ID feature,such as an IMSI, MS-IP or GTP-U-TEID, that is useable to identifypackets belonging to the user session to which the control plane packetbelongs.

In a block 812, if the control plane packet is a Create SessionRequest/Response packet, the S-vLB may use the at least one session IDfeature extracted in block 810 to generate a session ID feature vectorcomprising the extracted at least one session ID feature, and determinea status for the session associated with the packet and session IDfeature vector as “NEW”. In block 812, if the control plane packet is aModify Bearer Request/Response packet, the S-vLB may use the at leastone extracted session ID feature to update a session ID feature vectorpreviously generated for the session associated with the Modify BearerRequest/Response packet and determine a status of the associated usersession as “CURRENT”. In block 812, if the control plane packet is aDelete Session Request/Response packet, the S-vLB may determine a statusof the session associated with the control plane packet as “DELETED”.Optionally in a subsequent block 814, the S-vLB transmits to each NS-vLB101-102, optionally a MiniXDR, comprising data extracted and/orgenerated in block 810 from the control plane packet, for a new,current, or deleted user session.

When operating to SIT throttle and/or SILBA load balance packets inaccordance with an embodiment of the disclosure, NS-vLBs in Smart-vLBare configured to operate as stateful vLBs when processing to throttleand/or load balance user session packets. Each NS-vLB comprises asession monitor table in which the NS-vLB stores an identical record foreach user session that Smart-vLB 100 monitors and load balances and, asdescribed below, processes user session packets that it receives basedon data comprised in the monitor table. Each record of a session in thecopy of the session monitor table comprises the MS-IP address and IMSIof a mobile communication device that initiated the user session, andcurrent GTP-U-TEIDs associated with the user session. The record mayalso comprise an IP address for each of one or more vProbes 121, 122,123 (FIG. 6 ) to which all, or a same selection, of user packetsbelonging to the user session should be load balanced in parallel.

In a block 816 each NS-vLB optionally processes the data received in theMiniXDR from block 814, to determine if and how to update the copy ofthe monitor table that the NS-vLB comprises. The data, as noted above,may be data for a user session identified in the MiniXDR as a new,current, or deleted user session. If the user session identified in theMiniXDR is a new session, processing comprises applying a throttleconstraint to determine whether or not the identified session should beselected for monitoring and load balancing all or a selection of packetsassociated with the session to a same delegated vProbe for processing.To determine whether or not the user session should be monitored, theNS-vLB may hash the IMSI or MS-IP associated with the user session anduse a LUT that maps the hash to a decision to monitor or not monitor.Additionally, the NS-vLB may hash the IMSI or MS-IP and use a LUT thatmaps hashes to IP addresses of vProbes to determine the at least onevProbe to which packets belonging to the user session should beforwarded. IP addresses comprised in a user session record in themonitor table may change if vProbes used to process user packets fromcommunications network 20 are added, retired, or fail.

If the session identified in the MiniXDR is indicated as not being anew, but a current session, processing comprises determining whether thecurrent session is a monitored session or not, and if a monitoredsession, whether data in the MiniXDR warrants updating a record of thesession in the session monitor table. If the status of the sessionidentified in the MiniXDR indicates the session is deleted, processingcomprises determining that if a record of the session exists in thetable, the record should be deleted.

In a block 818 the NS-vLB operates in accordance with determinationsmade in block 816 to update the copy of the monitor table that theNS-vLB comprises. If the session identified in the MiniXDR is determinedto be selected for monitoring, a record for the user session is enteredinto the monitor table. As noted above the record may comprise an MS-IP,an IMSI, and GTP-U TEIDs associated with the session. In an embodiment,the record comprises an IP address of at least one vProbe to which userpackets associated with the session should be forwarded in parallel. Ifthe user session identified in the MiniXDR is a current monitored usersession and includes changes in data, for example changes in aGTP-U-TEID, associated with the user session, the record for the usersession may be updated to reflect the new data. If the sessionidentified in the MiniXDR is indicated as deleted, a record for the usersession if the record exists in the monitor table may be deleted.Following updating the session monitoring table the NS-vLB optionallyreturns to block 802 to receive another packet propagated overcommunication network 20.

In accordance with an embodiment of the disclosure, applying at leastone throttle constraint to determine whether a newly created usersession identified in the MiniXDR should be selected for monitoringcomprises, as noted above, selecting newly created user sessions formonitoring in accordance with a selection fraction. For example, if theselection fraction is 0.10, NS-vLBS in Smart-vLB 100 operate to selectsubstantially 10% of newly created sessions identified in MiniXDRsreceived from block 814 from an S-vLB. All the NS-vLBs apply theselection fraction in a same way so that they all select the same newlycreated user sessions for monitoring. Optionally, applying at least onethrottle constraint comprises applying at least one throttle constraintin addition to a selection percent. For example, a selection percent maybe applied after user sessions have been selected for a range of valuesof at least one of or any combination of more than one of an IMSI, IMEI,MSISDN, APN, QCI, and/or a CGI

If in decision block 804, if the NS-vLB determines that the packetreceived in block 802 is not a control plane packet but a user planepacket, the NS-vLB may proceed to a block 830 and extract at least onesession ID feature, such as an MS-IP and/or an IMSI, from the packetthat identifies a user session to which the packet belongs. In a block832 the NS-vLB may use the extracted at least one session ID feature todetermine if the user session identified by the at least one ID featureto which the user plane packet belongs is a monitored user session.Optionally, in a decision block 834 if the user session to which thepacket belongs is not a monitored user session the NS-vLB may proceed toa block 835 and discard the packet and return to block 802 to receiveanother packet. If on the other hand the user session to which thepacket belongs is a monitored session, the NS-vLB may proceed fromdecision block 834 to a block 836 and forward the packet to the at leastone vProbe designated to process all packets belonging to the usersession to which the packet belongs. From block 836 the NS-vLB mayproceed to block 802 to receive another packet.

There is therefore provided in accordance with an embodiment of thedisclosure, a load balancing system for a packet switched communicationsnetwork, the load balancing system comprising: at least one statefulload balancer configured to receive and process copies of control planepackets propagated by the network to determine a characterizing vectorfor each target feature of at least one target feature thatcharacterizes packets propagated by the network, which vector comprisesone or more characterizing components useable to associate packetspropagated by the network with the target feature; and at least onenon-stateful load balancer configured to: receive the at least onecharacterizing vector and copies of packets propagated by the network;use the at least one characterizing vector to associate a target featureof the at least one target feature with each of a plurality of thereceived packets; and load balance and transmit copies of packets thatthe non-stateful load balancer associates with a target feature of theat least one target feature to at least one probe, and copies of packetsassociated with a same target feature to a same probe of the at leastone probe.

Optionally, the at least one non-stateful load balancer is configured tobegin using components of the characterizing vector to associate copiesof packets with a given target feature of the at least one targetfeature only after receiving all the components of the characterizingvector for the given target feature. Optionally the at least onenon-stateful load balancer comprises a holding buffer and is configuredto store in the buffer packets associated with the given target featurethat it receives prior to using the characterizing components.

In an embodiment, the at least one non-stateful load balancer isconfigured to receive piecemeal the one or more characterizingcomponents for a given target feature of the at least one target featureand to begin using the characterizing components to associate packetswith the given target feature substantially at times at which thecomponents are received.

In an embodiment, the at least one stateful load balancer is configuredto transmit the one or more components of a characterizing vector to theat least one non-stateful load balancer using a high-speedcommunications protocol. Optionally, the high-speed protocol providestransit times for transmission of the one or more components from the atleast one stateful load balancer to the at least one non-stateful loadbalancer that is less than about 10 ms (milliseconds), less than about 5ms, or less than about 2 ms.

In an embodiment, the at least one non-stateful load balancer isconfigured to process copies of packets propagated in the communicationsnetwork to determine a characterizing vector for a target feature of theat least one target feature.

In an embodiment, the load balancing system comprises a library havingone or more selection filters that the at least one non-stateful loadbalancer may be configured to apply to select from among packets thatthe at least one non-stateful load balancer associates with a targetfeature of the at least one target feature, a subset of packets for loadbalancing and/or transmitting to the at least one probe for processing.Optionally, the one or more selection filters comprises a depletionfilter operable to select a portion of the received packets for loadbalancing and transmitting to the at least one probe and to drop packetsthat are not selected to limit a transmission rate to the at least oneprobe. Optionally, the depletion filter is configured to select a packetfor load balancing and transmitting based on a function of at least onetransmission rate upper limit and a transmission rate of the at leastone non-stateful load balancer. The at least one transmission rate upperlimit may be based on a capacity of the at least one probe to processpackets that the at least one probe receives from the at least onenon-stateful load balancer. The at least one transmission rate upperlimit may be a fraction less than one of a maximum packet processingrate of the at least one probe.

In an embodiment, the at least one transmission rate upper limit is adynamic limit configured to repeatedly be updated. Optionally the upperlimit is updated in accordance with a proportional-integral-derivative(PID) algorithm.

In an embodiment, the depletion filter operates to drop packets if inthe absence of dropping packets, the transmission rate would exceed theat least one transmission rate upper limit.

In an embodiment, the one or more selection filters comprises adepletion filter that has an activation packet traffic threshold andactivation delay and is configured to activate to drop packets at agiven time if packet traffic in the communications network is in excessof the activation packet traffic threshold for a period of time endingat the given time that is greater than the activation delay. Optionally,the depletion filter having the activation packet traffic thresholdcomprises a deactivation packet traffic threshold and deactivation delayand is configured to deactivate and cease dropping packets at a giventime after activation if packet traffic that the at least onenon-stateful load balancer receives is less than the deactivation packettraffic threshold for a period of time ending at the given time that isgreater than the deactivation delay.

In an embodiment, the one or more selection filters comprises adiscrimination filter that operates to select received packets for loadbalancing and/or transmitting to the at least one probe based onfeatures of user subscriber agreements under which users access andtransmits packets over the communications network.

In an embodiment, the at least one target feature comprises at least oneand any combination of more than one of an IMSI (International MobileSubscriber Identity), an IMEI (International Mobile Equipment Identity),an MSISDN (Mobile Station International Subscriber Directory Number),APN (Access Point Name), QCI, (Quality of Service Class Identifier), aCGI (Cell Global Identifier), and/or a flow in the communicationsnetwork to which a packet belongs.

In an embodiment, copies of packets that the non-stateful load balancersreceive are copies of user plane packets.

In an embodiment, copies of packets that the non-stateful load balancersreceive are copies of control plane packets.

There is therefore also provided in accordance with an embodiment of thedisclosure a method of processing packets propagated over a packetswitched communications network having a control plane, user plane, anda plurality of probes, the method comprising: receiving at least onecontrol plane packet associated with creating at least one user sessionin the network; selecting a set of user sessions from the at least oneuser session; determining at least one target feature that characterizespackets propagated over the network; and load balancing all packetssharing the at least one target feature that belong to a same usersession of the set of user sessions to a same probe of the plurality ofprobes for processing by the probe.

Optionally, the at least one target feature comprises at least one orany combination of more than one of a time period, APN (Access PointName), QCI, (Quality of Service Class Identifier), and/or a CGI (CellGlobal Identifier).

In an embodiment selecting a set of user sessions comprises selectingall of the at least one user session. In an embodiment selecting a setof user sessions comprises selecting a portion of the at least one usersession.

There is further provided in accordance with an embodiment a method ofprocessing packets propagated over a packet switched communicationsnetwork having a control plane, user plane, and a plurality of probes,the method comprising: receiving at least one control plane packetassociated with creating at least one user session in the network;selecting a set of user sessions comprising a portion of the at leastone user session; determining at least one target feature thatcharacterizes packets propagated over the network; and load balancingall packets sharing the at least one target feature that belong to asession of the set of user sessions to the plurality of probes forprocessing.

In an embodiment selecting a portion comprises randomly selecting usersessions from the at least one user session. Selecting a portion maycomprise determining for each of the at least one session an IMSI andselecting a set of user sessions based on the IMSIs. Selecting a portionmay comprise determining for each of the at least one session an MS-IPand selecting a set of user sessions based on the MS-IPs.

In an embodiment, processing the at least one control plane packetcomprises determining for each of the at least one user session an IMSI(International Mobile Subscriber Identity), an MS-IP (MobileSubscriber-IP) address, and a GTP-U-TEID (GPRS TunnelingProtocol-User-Tunnel Endpoint Identifier) useable to identify a packetbelonging to the user session. Optionally the method comprises using theIMSI and/or MS-IP determined for each user session of the at least oneuser session to determine an IP address for the probe to which packetsbelonging to the user session are load balanced. Optionally the methodcomprises for each user session of the at least one user session in theset logging a record comprising the IMSI, MS-IP and GTP-U-TEID and probeIP address into a session table. Optionally the method comprises usingthe records in the session table to load balance the packets.

In the description and claims of the present application, each of theverbs, “comprise” “include” and “have”, and conjugates thereof, are usedto indicate that the object or objects of the verb are not necessarily acomplete listing of components, elements or parts of the subject orsubjects of the verb.

Descriptions of embodiments of the disclosure in the present applicationare provided by way of example and are not intended to limit the scopeof the disclosure. The described embodiments comprise differentfeatures, not all of which are required in all embodiments of thedisclosure. Some embodiments utilize only some of the features orpossible combinations of the features. Variations of embodiments of thedisclosure that are described, and embodiments of the disclosurecomprising different combinations of features noted in the describedembodiments, will occur to persons of the art. The scope of thedisclosure is limited only by the claims.

The invention claimed is:
 1. A method of processing packets propagatedover a packet switched communications network having a control plane,user plane, and a plurality of probes, the method comprising: receivingcontrol plane packets associated with creating or modifying bearer foruser sessions in the network; processing the received control planepackets to determine at least one target feature that characterizespackets propagated over the network in the user sessions with which thecontrol plane packets are respectively associated; and selecting a setof user sessions from the user sessions associated with the receivedcontrol plane packets; for each user session in the set of user sessionsprocessing the received control plane packets to determine a sessionrecord comprising at least one characterizing feature of packetsbelonging to the session that is useable to associate the packet withthe target feature and identify the packet as belonging to the session;logging the session record into a session table using the records in thesession table to load balance packets sharing the same at least onetarget feature that belong to a same user session of the selected set ofuser sessions to a same probe of the plurality of probes for processingby the same probe; and throttling transmission rates by filtering ofpackets belonging to the selected set of user sessions responsive to afeature of the packets.
 2. The method according to claim 1 whereinthrottling transmission rates of packets responsive to a feature of thepackets comprises throttling transmission rates of packets responsive toat least one or any combination of more than one of a time period, APN(Access Point Name), QCI, (Quality of Service Class Identifier), and/ora CGI (Cell Global Identifier).
 3. The method according to claim 1wherein selecting the set of user sessions comprises selecting all ofthe user sessions associated with the received control plane packets. 4.The method according to claim 1 wherein selecting the set of usersessions comprises selecting a portion of the user sessions associatedwith the received control plane packets.
 5. The method according toclaim 4 wherein selecting the portion comprises randomly selecting usersessions from the user sessions associated with the received controlplane packets.
 6. The method according to claim 4 wherein selecting theportion comprises determining for each of the user sessions associatedwith the received control plane packets an IMSI and selecting a set ofuser sessions based on the IMSIs.
 7. The method according to claim 4wherein selecting the portion comprises determining for each of the usersessions associated with the received control plane packets an MS-IP andselecting the set of user sessions based on the MS-IPs.
 8. The methodaccording to claim 1 wherein processing the received control planepackets to determine the session record for each user session associatedwith the received control plane packets comprises determining an IMSI(International Mobile Subscriber Identity), an MS-IP (MobileSubscriber-IP) address, and a GTP-U-TEID (GPRS TunnelingProtocol-User-Tunnel Endpoint Identifier) for the record.
 9. The methodaccording to claim 8 and comprising using the IMSI and/or MS-IPdetermined for each user session of the set of user sessions todetermine an IP address for the same probe to which packets belonging tothe user session and sharing the at least one target feature are loadbalanced.
 10. The method according to claim 9 wherein determining thesession record comprises adding the probe IP address to the record. 11.The method according to claim 8 and comprising processing the receivedcontrol plane packets to determine, for each user session associatedwith the received control plane packets, at least one target featureuseable to identify a packet belonging to the user session selected froman MS-IP (Mobile Subscriber-IP) address, and a GTP-U-TEID (GPRSTunneling Protocol-User-Tunnel Endpoint Identifier).
 12. The methodaccording to claim 1 wherein throttling transmission rates comprisesusing a proportional integral derivative (PID) controller to control thetransmission rate.
 13. A method of processing packets propagated over apacket switched communications network having a control plane, userplane, and a plurality of probes, the method comprising: receivingcontrol plane packets associated with creating user sessions in thenetwork; selecting a portion of the user sessions associated with thereceived control plane packets; determining at least one target featurethat characterizes packets propagated over the network; and for eachuser session of the selected user sessions processing the receivedcontrol plane packets to determine a session record comprising at leastone characterizing feature of packets belonging to the session that isuseable to associate the packet with the target feature and identify thepacket as belonging to the session; logging the session record into asession table; using the records in the session table to load balancepackets that belong to the selected portion of the user sessions to theplurality of probes for processing.
 14. The method according to claim 13wherein selecting the portion of the user sessions comprises randomlyselecting user sessions from the sessions associated with the receivedcontrol plane packets.
 15. The method according to claim 14 whereinselecting the portion of the user sessions associated with the receivedcontrol plane packets comprises processing the control plane packets todetermine for each of the user sessions associated with the receivedcontrol plane packets an IMSI (International Mobile Subscriber Identity)and an MS-IP (Mobile Subscriber-IP) address.
 16. The method according toclaim 15 and comprising selecting the portion of the user sessionsassociated with the received control plane packets based on the IMSIsand/or MS-IPs.
 17. The method according to claim 13 wherein throttlingtransmission rates comprises using a proportional integral derivative(PID) controller to control the transmission rate.
 18. A method ofprocessing packets propagated over a packet switched communicationsnetwork having a control plane, user plane, and a plurality of probes,the method comprising: receiving control plane packets associated withcreating or modifying bearer for user sessions in the network;processing the received control plane packets to determine at least onetarget feature that characterizes packets propagated over the network inthe user sessions with which the control plane packets are respectivelyassociated; and selecting a set of user sessions from the user sessionsassociated with the received control plane packets; load balancingpackets sharing the same at least one target feature that belong to asame user session of the selected set of user sessions to a same probeof the plurality of probes for processing by the same probe; throttlingtransmission rates of packets belonging to the selected set of usersessions responsive to a feature of the packets; processing the receivedcontrol plane packets to determine, for each user session associatedwith the received control plane packets, an IMSI (International MobileSubscriber Identity), an MS-IP (Mobile Subscriber-IP) address, and aGTP-U-TEID (GPRS Tunneling Protocol-User-Tunnel Endpoint Identifier);using the IMSI and/or MS-IP determined for each user session of the setof user sessions to determine an IP address for the same probe to whichpackets belonging to the user session and sharing the at least onetarget feature are load balanced; and for each user session in the setof user sessions, logging a record comprising the IMSI, MS-IP andGTP-U-TEID and probe IP address into a session table.
 19. The methodaccording to claim 18 and comprising using the record in the sessiontable to load balance the packets.